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(54) Title: METHOD, APPARATUS AND SYSTEM FOR VERIFICATION OF PATTERNS 
(57) Abstract 

A method apparatus and system for verifying the establishment of a pattern includes means for storing a template of the pattern, 
means for acquiring an image of the established pattern, a first image processing means to alter at least one of said template and acquired 
images to produce at least two resultant images, comparison means to compare the two resultant images with the other of said template and 
acquired image and means to evaluate the results of each comparison to determine if the established pattern includes a defect. Both gray 
scale and binary processing and comparisons are disclosed for use as required. 
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METHOPi APPARATUS AJND SYSTEM FOR VERIFICATION OF PATTERNS 
Field Of The Invention 

The present invention relates to a method, apparatus and system for the automatic verification 
of patterns established on a substrate or other surface. More specifically, the present invention 
5 relates to a system and method for determining if an established pattern contains one or more defects. 
Background Of The Invention 

In many fields, such a printing, it is desired to determine that a pattern has been properly 
formed on the substrate or other target surface. For example, when printing labels for 
pharmaceuticals, it is desirable to insure that dosage information has been properly printed on the 

10 label. Specifically, a defect such as one which resulted in one or more labels having a missing 
decimal point which would otherwise change 1.0 mg to 10 mg, with perhaps serious and/or fatal 
results, is unacceptable. Similarly, for any high quality printing such as postage stamps or security 
printing for lottery tickets, etc., it is important that the printed pattern be correct within preselected 
bounds. This is also true for other patterns such as printed circuit boards, semiconductor device 

15 features, flexible printed circuits, etc. 

Prior art verification of printed patterns has been performed either post production (i.e. - 
after the pattern establishment process has been completed) or in real-time (i.e. - while the pattern 
establishment process is being performed). Post production verification has disadvantages in that a 
great deal of wastage can occur as there is no immediate feedback to the pattern establishing process, 

20 to correct the process in view of detected defects. Thus, if for example a process error occurs in the 
middle of a production ru:\ half of the production run of finished product may include defects and 
have to be discarded. Further, post production verification requires an additional, separate step and 
therefore results in increased labor costs and increases the time required to complete a production 
run. 

25 Real-time verification systems are therefore often preferred over post production systems. 

However, prior art real-time verification systems suffer from their own problems in that, when high 
confidence is required in determining errors, many false indications of defects are produced as no 
means is available to distinguish between actual defects and acceptable variations due to normal 
pattern establishment process variations. In some circumstances, when high confidence is not 

30 required, the occurrence of false indications of defects in prior art systems has been lowered by 
reducing the sensitivity of the pattern verification system, but this necessarily results in reduced 
quality control of the final product. 

Further, due to limitations in the speed with which prior art systems could perform 
verification, such systems often required the limitation of the production speed of the pattern 

35 establishment process to less than the available maximum processing speed. 

It is desired therefore, to have a pattern verification system which can verify the 
establishment of patterns to a desired degree of accuracy, accommodate normal process variations 
without indicating false defects and which can operate with relatively high speed pattern 
establishment processes; 

40 Summary Of The Invention 

It is an object of the present invention to provide a novel method, apparatus and system for 



WO 97/36260 



-2- 



PCT/CA97/00196 



determining if an established pattern contains a defect which obviates or mitigates at least one of the 
disadvantages of the prior art. 

According to a first aspect of the present invention, there is provided a method of 
determining if an established pattern contains one or more defects, comprising the steps of: 
5 (i) defining a digital template image representing a desired pattern; 

(ii) acquiring a digital image of an established pattern; 

(iii) image processing at least one of said defined digital template and said acquired digital 
image to obtain at least first and second resultant images; 

(iv) performing at least one pair of comparisons comprising: a first comparison between the 
10 other of said defined template and said acquired digital image and said first resultant image; and a 

second comparison between the other of said defined template and said acquired digital image and 
said second resultant image; 

(v) evaluating the results of at least said at least one pair of comparisons to determine if said 
established pattern contains one or more defects. 

15 According to another aspect of the invention, there is provided a system to determine if an 

established pattern contains oneor more defects, comprising: 

means to store a defined digital template for said pattern; 
means to acquire a digital image of said established pattern; 

image processing means to process at least one of said defined digital template and said 
20 acquired digital image to obtain at least first and second resultant images; 

means to perform at least one pair of comparisons comprising a comparison between the 
other of said defined digital template and said acquired digital image and said first resultant image 
and a comparison between the other of said defined digital template and said acquired digital image 
and said second resultant image; and 
25 means to evaluate the results of said comparisons to determine if said established pattern 

contains one or more defects. 
Brief Description Of The Drawings 

Preferred embodiments of the present invention will now be described, by way of example 
only, with reference to the attached Figures, wherein: 
30 Figure 1 shows a block diagram of a prior art pattern verification system; 

Figure 2 shows a representation of a template image; 

Figure 3 shows a representation of an image of a pattern to be verified; 

Figure 4 shows a representation of a prior art subtraction technique to identify defects in a 

pattern; 

35 Figure 5 shows a representation of images of four possible configurations of an example 

pattern; 

Figure 6 shows a representation of a prior art subtraction technique to identify defects; 
Figure 7 shows an array of digital values representing an image of a pattern; 
Figure 8 shows a representation of a median rank filter being applied to one of the digital 
40 values of Figure 7; 

Figure 9 shows a representation of the result of the processing the array of Figure 7 with a 
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median rank filter; 

Figures 10a through lOd represent possible outcomes of a dark feature missing comparison- 
figures a teoogh m represent possible outcomes of a light feanue missing comparison-' 
figures 12a through 12c reprint possible outcomes of an extra dark feature comparison- ' 
figures 13a through 13 c represent possible outcome, of an extra light feature comparison 

,™ h 8 " re . " W ° Ck " ia8ram ° f ' Prefemd of a gray scale system in 

accordance w.th the present invention which employs four comparisons 

Figure 15 shows the system of Figure 14 as implemented in commercially available 

Cyberscan® modules; 

an examp,e Tempiate image for - * « * — - ~ 

Figure 16b shows an example pattern image to be verified with binary comparisons- 
Figure 17 shows a graphical representation of the results of a first binary comparison- 
F.gure 8 shows a graphical representation of the results of a second binary comparison 
F.gure 19 shows a graphical representation of the result, of a third binary comparison- 
Figure 20 shows a graphical representation of the results of a fourth binary comparison- 
Figure 21 shows a graphical representation of the results of a fifth binary comparison- ' 
F.gure 22 shows a graphical representation of the results of a sixth binary comparison; 

20 r 8Ure 1 f WS 3 8raPhiMl rePreSentat '° n ° f the rWU,ts of * — th binary comparison; 

F.gure 24 shows a graphical representation of the results of an eighth binary comparison- 
F lg ure 25 shows a graphical representation of the results of a ninth binary comparison- ' 
Figure 26 shows a graphical representation of the results of a tenth binary comparison- 
F.gure 27 shows a block diagram of a preferred configuration of a binary system in ' 
accordance with the present invention; and 

25 cfijszzr to of Fieure 27 u - — * «** 

-P M »itelP«cijpdMOf The Invention 

<■«- F0 ^ C ! a s ri,, • ' Pr '° r Verif,Cati0n SySBm wi " fira be before detailed «, 

30 fZ 1S ' in ™' i0n - "^H™"^^ generally a, 10 in ^ 

As used herem. the term pattern is intended comprise any text, schematic, graphics, bV 

ZZ^T^'^'T 3 ^"^-y -era Also, as u!ed here „ the P ^ 

aZt ™?^*&t!!!SBe!aJB^. mounting or otherwise causing a 
pattern to be formed is referred to as establishing a pattern 

array tfTL'T '7 T" " " " 4 "~ d " ta * 22 in of an p 

amy of 8 b„ p,xels, me value o^chjix£ represemin e the degree of darkness in the acquired ^ A ' 
d.g.ttzed rniage wijhOb eing uSS^tolx*^ ^- ' qU ' red ■ ^ 

- „„„H ■ T " eSlred Patle^, 1,38 ta di 6 teKl "0«d in image memory 26 His 

ZrM* 51155 ^^^^^^!^^- Asuitable 
processor 30 tan chares, as described below, tiHSJEtaiTOnK,,, 26 ,„ the 
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acquired image stored in digitizer 22 and produces an output to output means 34, which may be a 
video display, an error annunciator or any other suitable output device as will occur to those of skill 
in the art. 

Specifically, the comparison process proceeds as follows. Figure 2 shows a representation of 
the template image stored in image memory 26 and Figure 3 shows a representation of an image 
which has been acquired by camera 14 and digitized and stored in digitizer 22. It will be understood 
by those of skill in the art that, while these images are shown in the Figures with the pixels being 
represented with filled blocks, in fact the digitized images are stored as an array of digital values 
which may be operated upon mathematically by processor 30. 

A reference fiducial 50 is included in the template image and a corresponding reference 
fiducial 54 is included in the pattern wh ose image was acquired bv ca mera 14. These reference 
fiducialsHejisedjoalign the images so that theyare spatially coincident . In example of Figures 3 
and 4, assuming that the upper left hand pixel ofladi Figure is indexed 'as (0,0), the center of 
fiducial 54 is located at (4,2) whereas the center of the fiducial 50 in the template image is located at 




K n n A , w •" " iV - wmpiaic image is iocatea 

15 (3,1). ^oi^^o jnate the images spatiallyjo incid en t, the acquired image is shifted left one 
v , pixel and upwards one pixel by processor 30. ^> 3f ^( A^fej Q, fa c / </, 

Once the template and acquired images are spatially coincident, a simple subtrac tion ' ^ 
.operation is performed to Megifrdgg.. As shown in Figure 4, in an ideal situation any non-zero 
valued pixels in the result of this operation n*tMjsSuii&toK^to&. Specifically the 
20 two non-zero pixels 58 in the result are the two defects in thTa^eTim^T These two pixels'can 
be output to output means 34 or used in any other manner to report the detection of defects in the 
pattern under consideration. 



Unfortunately, tbjLpiipjLarjsystem described above has serious drawbacks when it is applied 
to many typical pattern establishment processes. One of the main problems with such prior art 
^ystems is that most processes for establishing a pattern are not perfect and experience some variation 
^jhe established patte rjUKhic h is not considered to be ^defet. For example, in printingpr^eT 
. the positioning of various features willoft^^ and/or t0 each . 

^B^^Sm^SS^^^- F °r sample, Figure 5 shows four images including three 
images 60a, 60b and 60c wherein the positioning of the deci mal point has drifted relative to the 1 and 

~& v CS and ° ne " nage 60d wherein decimal P° int has not been printed. " ~ 

^ W3 ^ If thC above - descr *ed prior art system 10 is employed to verify the patterns of images 60b, 
-3 ? £ 7 ' T d 6 ° d ' USing 603 aS a tem P late ' ana,vsis of ea <* of images 60b, 60c and 60d will result 
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* • j » . ' — gvo uuu, uuv. auu ouu win result 

^defects being reported to_o i m Z unr*ai^ > even though only image 60d actually contains a defect 

* ■ 'o x *A U l ' miSSmg deCima, P ° inl - F ° r CXamplC ' thC ° UtCOme 0f the subtraction operation-type analysis 

~N / c* 0511,8 603 M 3 temp,ate iS Sh0Wn in Figure 6 wherei " the resultant 64 indicates 

' ^multiple fabe defeett 68. 

. The present invention will now be described with reference to Figures 7 through 27 The 

^present inVention js °P erab,e 'ojfet^^ reducj Qr eyen 

^ l ™ a ^^^ be pro6u ^ ^^ t oc ess variations in 



itn u uv " ^ 1 — " w ,v ,IW| »<" V* uucos variation: 

WJ^abtehm^ In the immediacy following toliiTon, a printing pro^sTwliTbe" 

Ud.scussed as a representative example of a method of establishing patterns. It will be appreciated 
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however, by those of skill in the art, that the present invention i s not limited to use w ith printed 
x ^auerns and can in fact be used with a wide range of pattern establishment processes indu^Tm 

• llmitCd t0 «»* etc hing, gravurep rinting, sputtering of layers, mechanical placements 
eonjEgnentSi etc. , , n * . , —j- 4- 

In the present invention, Jgatus gs of a pattern are allowed to drif t , due to normal process ] 
Rations, within predefined bounds. Itowemrd^fy^^^ot^^p^^ 
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.^Dds^f the features are malformed or missing, ^T^^^^^^^^t will. 
*\pcr jjcdcemcd jo have occurred . 

«A in ,,. ! n °" e Preferred embodiment whi <* operates with gray scale images, the present inv.nrinn 
^ 10 ut.hzes four separate comparison operations, described below. When processing speed is of interest 
6 ^ such as m the real-time monitoring of a high speed printing process, these comparison operations ' 
may be conducted in parallel. While, as is discussed below, the outcome of a properly se.ected pair 
of comparison operations can provide adequate defect detection, it is believed that the outcome of 

15 dr ec T PerIy SC,eCted PaifS ° f COmPariS ° n ° Perati0nS Pr ° VideS *" additi ° nal adV3ntage in detectin * 

The present invention makes use of known image processing techniques. Specifically, in the 
presently preferred^gra^s cale embodim ent of the instant invention, discussed below with reference to 
Figures 7 through 15, a form of image processing which is referred to as gray scale morphology is 
employed. Further, in the presently preferred embodiment, the gray scale morphology operations 
20 are tmplemented with rank value fil ters,. As will be apparent to those of skill in the art, the present 
mvention is not limited to rank value filter-based gray scale morphology, and any other suitable 
•mage processing technique and/or implementation may be employed. 

In the presently preferred binary embodiment of the instant invention, discussed below with 
reference to Figures 16 through 28, the preferable image processing technique which is employed is 
25 a form of correlation filtering, specifically binary correlation, although any other suitable binary 
image processing technique may also be employed. 

Rank value filters, such as those used in the gray scale morphology embodiment, operate on 
.each pixel m an image byra^ng^ ^s of thepixels in a neigh borhood (kernel) centered on ,he 
, n T^'"* P""^ referred tol^^ ^^ The s^fli^e^ 
30 referred to as the size of the kernel, can be defined as R rows by C columns and the resulting value 
of the target pixel is calculated by taking all of the unprocessed pixel values in the R by C kernel 
centered on the target pixelj^s^j^em. The resulting value of the target pixel, in a filter of 
rank N „ set to the value of the Nth ranked pixel. Depending upon the value selected for the rank 
^ N, vanous filter effects may be obtained, including a minimum, maximum and/or median filter 

^A? -valued n m ? f 7 " Ci8ht ^ Ci8ht ^ re P re ^on of an image which is mostly dark (low 
■^Ji y&vmM* whtch has a light feature centered at (4,3), assuming that the upper left tZr 

tltl : S T ^ WHiCh light spots. Figure 8 demonstrl 

he apphcanon of a rank value filter with a three by three kernel and N=five, when applied to a 

40 IlTrlr? 1 (4 ' 3> " FigUrC 7 2 threC ^ kerne "' «*« N = 5 in a 
40 medtan rank filter function as the ranked fifth value of the nine values is the median value As 

shown, the pixels in the three by three kernel centered on (4,3) range in value from 21 1 to 0 with the 
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fifth ranked value being 175. Thus, the resulting value for the target pixel is 175. ^^>V\V S 

The complete result of applying thejnedianranl^^ filter to the image of Figure 7 is \^p]s^\ 
shown in Figure 9 wherein it can be seen that the random light spots have hern Himin^ H and the 
size and lightness (i.e. - the value of the pixels) of the * + 1 feature have been reduced, although its 
5 shape and positioning have not changed. 

With N set to the highest rank, a maximum rank filter is obtained. When applied to a gray 
scale image, a maximum rank filter results in the erosion (i.e. - thixming out) of dark fcatures and ^ 

dilation (i.e. - fattening up) of light features and is referred to herein as a "dilation" or "dilate" 
operation. 

10 Conversely, with N set to the lowest rank, a minimum rank filter is obtained. When applied 

to a gray scale image, a minimum rank filter results in the dilation of dark features and the erosion of 
light features and is referred to herein as an "erosion" or "erode" operation. 
C ^J&^ ^ will be understood by those of skill in the art that the descriptions above refer to a gray 

Q, ^ SCa ' e value s y^em wherein dark image pixe ls are low valued and light image pixels are high^alue7 
yC^-AS In the reverse system, wherein dark image pixels are high valued and light image pixels are low 

valued, the above-described effects of%jnaxjrm^ ^ rank fi iters are reversed t ke . - a 
^vi^ minimum rank filter will dilate light features and erode dark features and a maximum rank filter will 
sj^/^ erode light features and dilate dark features. However, as used herein, the term "er ode" or 

"erosion" refers to the erosion of light features and the term "dilate" or "dilation" refers to the 
20 dilation of lightJeaMgiandjh^^ features . It will also be understood by those 

of skill in the art that grayscale image pixels needjimbe j-bit values ando ther larger or smaller 
values, as appropriate to the particular application of the present invention, may be employed as 
desired. 

In the presently preferred gray scale embodiment, the instant invention employs four 
25 comparison operations which are presently believed to be advantageous for use in identifying defects. 
These comparison operations are intended to detect: 'extra dark feature' defects; 'missing dark 
feature* defects; 'extra light feature* defects; and 'missing light feature' defects. Each comparison 
will be described in turn below. 

The comparison to identify a missing dark feature defect can be represented by 
30 LiveaioDE * Template^ 

where Liv eER0DE is the acquired image which has been processed with an appropriate morphological V 
filter (for example , a minimum rank in the case of light valued p ixels having greater values than dark ^ 
valued pixels), Template RAW is the non-processed template image and A ( is an image intensity 
J|*ghojdjvIu^ value in ft,, temp|ate Aj is described jn m0fe 

35 detail below. 




In this comparison, as in all of the other comparisons described below, jemjplate,,^ can be a 
jggglgjejvjnch^^ be an image wh |^ nas - been 

"-^reyjojisjy^^ For example, in some circumstances Template RAW can be the unprocessed live 
image (Live^w) acquired during the last verification operation. 
40 As will be apparent to those of skill in the art, in this missing dark feature comparison and all 

^AA^ thC -^ her com P arisons described below, the size of the kernel defines the bounds about the 
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template position within which a feature must be found or else a defect is detected. For example, a 
three by three kernel, such as that referred to above with respect to Figure 8, allows^f eature-to^drift 
^upjo one pixel in any direction from its position in the template. Therefore, defining the size of the 



. 

V^/*\ kernel used in the rank filtering wjU Jefme the desiredj i mits for variance in the established -pattern , 

- tpw\ 5 Of course, it is possible to define different kernel sizes for different regions of the pattern to be ^ ° ( 
verified. For example, in the case of the pharmaceutical label mentioned above, a three by three 



\3 



kernel can be employed when image processing a region of the label image which contains dosage 
information and a five by five kernel can be employed when image processing the remainder of the 
label. Further, processing with the three by three kernel may proceed in parallel with processing with 
10 the five by five kernel or in seriatim, depending upon the particular hardware implementation. 

As will be apparent to those of skill in the art, the kernel need not be square and thus the 
limits for variance can be defined separately for horizontal and vertical directions. For example, a 
rectangular kernel can be employed if feature drift is acceptable to a larger extent in one direction 
(e.g. - V or horizontal) than in the other direction (e.g. - y or vertical). Also, non-rectilinear 
15 kernel shapes may be employed. For example, a generally circular kernel can be employed when 
feature drift in a radial manner is acceptable. 

In the representations shown in Figures 10ajhr oughJ3c. and described belo w, asspat-ially^. 



y 0^ * -p; — w» v » s » aim uca^nucu oeio w, assparaaiiy*^ 

^ CQ^s^pnding line of pixe l s in each of the acquired-ima^nd the template image, at least one of/ ( 
Cf*'\ * ^*** asteen ima 8 e Processed by a selected morphological filter, is being compared. In thesetne * 
^W^?? dimensional representations of the p rocessing of the two di mensional images, the heights of the traces I 
5PrTvv\|'/\ reflect the value of the respective nixels. with higher trarpc rx,»h,.«\ ;~a:~~.:-~ . ' 



jf^i reflect ^ va,ue of he respective pixels, with higher traces (values) indicating lighter pixels, 
fl The datums for the two traces are snae«H hv;th»M«i,>a-A . ..,w:„u _n r__> j-rr- 



The datums for the two traces are spaced bvahe^alue^whirh allows Jo& a.predefined 
WtMg fctowtotoa in the intensity of the pattern ; In a preferred embodiment oftoe present 

^invention, A, varies for pixels rwithin the tempj^ Specificaily ^aluetfor^is^predetermined for 

► Leach noccihU nivp 



4\ 
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pach possible pixel- value using a -logarithmic funetion. T his simulates the operation of the human eve" 
jnthat A; is larger fojibrighter pixels and smaller for darker pix els. In such a case, a larger change 
in intetKitvJnJ>rj^^ ^ me index 

V for A, is merely the corresponding pixel value for which the respective A, has been predetermined 
f As will be apparent to those of skill in the art, other techniques of implementing A, can also be 
30 favorably t m p\oyt^ g ^ e ^t i ^^^^^^ tm ^ m ^ ^ ^ va , ues and 



Lo£sujrounding pixels,, etc. 



In these comparisons* an acquired image which exatt.|^6rresp0nds»to A me^template«mag.^ 
^Represented by an identical pair of traces, vertically spaced by the value A, A non- 
.dentical but defect free pattern is represented by any pair of non-intersecting traces and a pattern 
with one of more defects is represented by a pair of traces with at least one intersection point. It will 
be understood by those of skill in the art, that the representations in these Figures are only intended 
as a convenient means to convey the operation of the comparisons described herein and that the 
comparisons are in fact performed mathematically and need not be performed on lines of pixels but 
-nstead can be performed in any sequence provided that pairs of spatially coincident pixels of interest 
40 are compared. 

Figure 10a shows a representation of the missing dark feature comparison wherein the dark 



WO9TO6260 

PCT/CA97/00196 

-8- 

Tta - apparent as there is „ 0 intersection of fc ^ ^ 

« « f i rr pixel5 dark frawre in 11,6 «**- " — *» 

area of the Template wee, wh,ch represents the pix ,|s „f u>c dark tan i„ the Template. As wiU 
5 be apparent ,o those „, skUI in the art, the widening „f ,„e iowered area of the LiveJL .race int 

f 77k V ° f eroSi °" ° f li8h, <""■ te -"—8 Z the da k 

feature) by the morphological filler. 

Figure 10b shows a represent of the a™ comparison wherein the dark feature in ,he 
Figure 10c shows a representation of* sane comparison wherein rhe dark feature in the 

z^r^ f from v** * m a ™ u « *« « ™ 

size and thus a defect is detected, as indicated by the shaded, overlapped, portion 

15 h mi-in^r 'I" ^ 3 rePrCSe '" a ' i '" 1 ° f » — """P"*"" ^ - ** feantre 

15 -f^a^ aC< " afed "gig*?* — " *» U«— trace) and thus aS" 

detected, as indicate d by the shade d. overl^rpT;^ 

The missing light feature comparison can be represented b_y 

L>*uun 2 Template*,, (*r) ) 
20 dT™ V tT ^ r"*" 1 "" age Wl " Ch "* • morphotogica. filter which 

desenbed above. F,gure 1 la shows a represenution offe VB&ZZSSSfi&mn is 

> '" " ^ *" 001 drifttd a ° d *" "° defKt * *«• ' 1" *ows a 

Jtf « 1 S ^ iS °" * ll8h " ature » P»«H» 'be acquired taagea^ 

,, ^S£^liteemE!?te£«ito by an amountlessj han one half the ten,,! mu^JS - 
2^ Waged. Figure . le shows a representation of this comparison wherein the light feature is present 
■ -he has drifted from me template position b, an amount greater man one half 

^e*ernel-size and Bins a defect is reported* Finally. Figure I Id shows a represenution of this 
comparison wherein the light feature is missing from the acquired image and thus a defect is 
reported. 

30 The extra dark feature comparison can be represented by 

. T . Uve "" * Tem P!mE*oDem&if 

21 UnPr ° CeSSed aCqUirCd Tem P ,at W is the template image to which an 

f,,ter ^ ^ aPPHed l ° Cr0de thC H8ht featUreS and - * value, as 
35 I! < 8UrC Sh ° WS 3 re P resentation of «* comparison wherein an expected dark 

feature „ found in the acquired image at the template position and thus no defect is reported. Figure 
T2b shows a representation of this comparison wherein an extra dark feature is detected in the 

Tl 6 r *" 2 defeCt " reP ° rted - Fi8UrC 12C Sh ° WS 3 repr ~ n of < his comparison 
wherem a ark feature » detected in the acquired image at the template location, but it is too dark 
(i.e. - black rather than gray) and thus an error is detected. 
40 The extra light feature comparison can be represented by 

Uve MW i Template DllAn + A, 



WO 97/36260 

PCT/CA97/00196 

-9- 

where Live RAW is the unprocessed acquired image, Tempiat W is the template image to which an 
approbate morphological filter has been applied to dilate the light features and A f is a delta value s 

r ; F r e 13a shows a representa,ion ° f ** ***** - ~ z 

eature ,s found » the acquired image at the template position and thus no defect is reported. Figure 
5 13b shows a representation of this comparison wherein an extra light feature is detected in the 
acquired image and thus a defect is reported. Figure 13c shows a representation of this comparison 
wherem a ,, gh t feature is detected in the acquired image at the template location, but it is too ,i ht 
(i.e. - wlrne rather than gray) and thus an error is detected. 

.0 identifv f C< "" Cra,>la,ed *" Mher """P™ « »»> »= -Ployed with Ihe presem invention to 

TZZT or types defcm prcva " m in ~ r ^ P~ 

ror example, the comparisons 

£iv«i>iurii Template^. A, 
are sttmlar ■„ the extra dark feature and ex.ra light feature comparisons given above but allow for a 
15 greater degree of drift ofthe pattern. The selection of other suitable comparisons 

^tr^rL^r presen ' invcntion wui ^ apparen ( - - — - - - - - 

■y. The present inventors have determined to, by performing at least one appropriate ^fW* 

2oCT? J*-"*"**' ^^ M^ good pa^rn verif.ca.on 7 T ^ 
^^aSSEEfa^. as will be ^^r^^f^^rTor dark 

pauen* established on a light substrate ™ appropriate corresponding pair is the extra dark feate 
compar* „ and the missing dark feature comparison. Once the comparisons have been performed 
tor results can be combined through a logical OR to identify defects 

25 ~~.8P.ir Of colons is the extra light feature comparison a™, the missing light fire 

ZTT In"; 0 "" 1W ° COmP " i!<, " S ta « ^-=d. their results can be combined 
through a logical OR to identity defects. 

^ oerfoJT"" COrreSPOndi " g "* Prides good pattern verification 

™ssing Ugh, feature comparison 104, extra dark feature comparison 108 and missing dark feature 

TZT 1 1 a " performed - A " < "" pu ' 1 16 E ™ *— »— - £ ™ 

IT T f ° U ' P, " S ° f ** 108 - «•» "8", feature comparison 

r:;r;oR ,r mg ,igh ' fm,re 104 ^ ** *- ^ -5 

While the aboveKlescribed combination of four comparisons is presently preferred it will be 
pparen, to those of skill in the art that active configurations can be empje P ^ L 

comparison 100 and missing light feature comparison 104 through a logical OR operation and to 
combine the output of extra dark feature comparison 108 and missing dark fea JHC^ U2 
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through a logical OR to obtain outputs indicating Light Feature Defects and Dark Feature Defects 
In some embodiments of the present invention, the Template d ata will not change d uring a 
pattern verification process. Accordingly, the processed Template data' (i.e. - Template ER0DE or 
Template DILATC ) may either be derived as required, as described above with reference to Figure 14 or 
5 pre-computed once and stored for use as needed in a suitable memory. Also, as will be apparent to 
those of skill in the art, for clarity the buffers or other associated functions required to process the 
s,gnals have been omitted from Figure 14. For example, if required the Live RAW signal can be stored 
m a FIFO buffer before being transmitted to Extra Dark Feature Comparison 108 so that the 
Template ER0DE data and the Live RAW data arrive at Extra Dark Feature Comparison 108 
10 simultaneously. 

The implementation of the presently preferred ^ scale embodime nt the present ^ 2?% 
inve ntion described above is not particularly lim ited and implementations in dedicated hardware ^ 
general purpose computers and/or digital signal processors are contemplated. The present inventors 
have implemented the gray scale embodiment of the present invention with various Cyberscan® 
15 modules, as shown in Figure 15, which are commercially available from Focus Automation Systems 

5 aO^ * 3 " 554 ParkSidC DriVC ' Water, °°' 0ntari0 ' Canada - N2L 524 ^d which are described in U S 
(% \ ^fttarSj^TO^ In the Figure, eight Cyberscan* modules are employed, and specificall^ 

system comprises: a CAM-DDC3-00 camera module 300; three BUF-1000-00 buffer modules 304, 
308, 312; a GMF-2000-00 gray scale morphology module 316; a BMF-8000-00 binary morphology 
20 module 320; a LUT- 1000-00 look up table module 324; and a PCD-1000-00 display module 328. In 
this implementation, the Live RAW image, is acquired by a suitable camera 332 and any defects 
detected by the system are displayed to an operator on a conventional monitor 336. 

It should be noted that one of the perceived advantages of the present invention is its 
susceptibility to implementation in parallel hardware to provide relatively high speed pattern 
25 verification /defect detection. For example, in the embodiment shown in Figure 14, each 

morphological niter_£an be implemented by a separate processor, as can each feature comparison. If 
additional speed is required, two or more processors can perform subsets of each morphological filter 
function or comparison and the depth of the data pipeline can also be increased as desired. 

As will be apparent to those of skill in the art, the embodiment of the present invention 
30 described above is not limited to gray scale patterns. Color patterns can be verified by performing 
the above-described process on each of three acquired images, each image having been acquired by a 
separate camera equipped with a respective appropriat e color filter ( for example - Red, Blue and 
Green). Alternatively, a color camera can be employed to acquire a color digitized image in an 
appropriate format, such as an interleaved bitmap, etc. and an appropriate digital colorspace 
35 conversion operation (Hue, Saturation and Brightness (HSV) or Luminance, Chrominance r and 
i Xhrommance b (YCrCb), etc.) can be performed to provide three Live RAW image data streams. 
^ In some pattern verification applications, it is possible to determine the presence or absence 

. V of a feature using a binary template and an acquired binary image. For example, in a printed circuit 
toKdve^ is possible t o arrange the lighting, camera, and optics of the image 

~ flj#° aC£ « u,nn 8 s y stem »«* th at, in the acquired image, the conductive (copper) traces appear pure white 
NO and the circuit board ^ajfctrateap^^ In such a case, the image can be n^SSidV"" 
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class of d.g,u, images which may havepixels represented „ y . ^^^^ ^ 

estab.tshed pa-™ represented bv . binlry ^ bjmty - f 
processing operation. ,„ a hinary correlation process, . ^ of prKe J^^^ 

J*^ °" er te -g3£g^g!lj!' v»te °f all of the image pi xels within ,h,T ■ 

\ /w « o Thic c» m T — — ""age pixels within the kerne are summed 

J^io p?xe,r h h " ^""^Ndy, ! , ,, .o determine the new value for «JZl 

^ 10 pixel w „h tne target pixel being « » , if fte „ exceeded ^ ~ « 

-pie. 0,= Uphold „ set ,„ „ro, a btary ^ „ .^^f! T ^ 

the selected kernel (for example a 7x7 kernel has a maximum value of 49) a binan- erosion 
operation is performed (e e - TemnW, „. i c "< a omary erosion 

15 values (i.e ^^^Z^T^ ^'^^ 0f °'»e' ."reshold 

corre.at,on operate may be employed as will occur to those of skill in the art 

scale morphology and the processors required to imDlement if * . nangray 

•» those required for gray scde morphology lKS 

/ defect d^ion blnMy emb0dtaM "' ° f inVeMi °"' «* «— - 

Temnlai. ^ . u- ^ Livea,„ 8 , Template,,,,, Templateo™, and 

*" mUlr '° ^ deSCribed ab ° v,! '» '""ence » dJZ scale 
embodiment, the binary comparisons described herein can also em„lo„ . • 

30 Ita Live^ data is obtained by performing an eroo Tl ^ ^ W " ^ 

- *en performing a dilate coLLon opL^ Z 2, " STT °" 1 ^ *"* 
performing a dilate correlation operation « the u^Z^^T ' " * 
operation on U.resu,,. A. -» b. ^ » «* 

operates to open o, 'break apart' two obje* joined by an 1^11^ ™ 
35 processing operates ,„ close or •fill in' holes in solid objects. 

In the presenUy preferred binary embodiment of the present invent™ ,u . ■ 
Templates images are obtained b, performing a maxinZT We 
threshold binary correlation „n.„, u mM ™. « desired) 

u uinary correlation operation on the correspondins RAW m'ffnak *«a »h . • 

Template,,^ images are obtained b, performin, , JT , 
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employed as will occur to those of skill in the art. 

The ten binary comparisons which can be employed in the presently preferred binary 
embodiment of the present invention are listed in Appendix A, as are the types of defects for which 
they are perceived to be useful in detecting. It is presently contemplated that the kernel sizes selected 
5 for each comparison would be the same, although it will be apparent to those of skill in the art that 
this need not be the case and different operations can employ kernels of different sizes as desired 
and/or appropriate. As will also be apparent to those of skill in the art, in cases wherein the 
foreground is merely the inverse of the background of an acquired image, some of the ten 
comparisons will be redundant. In the more general case wherein the foreground and background 
10 images may have been acquired with different binarization thresholds, there may be no redundancy 
of the comparisons. 

Verification of patterns / detection of defects with binary comparisons is accomplished in the 
same manner as the above-described verification with gray scale comparisons, with the comparison 
data sets being spatially aligned and then processed by appropriate correlation or other filters as 
15 required and the selected comparisons being performed. The results of the comparisons are then 
evaluated to determine if defects are present. 

Figure 16a shows an example of a template image for pattern verification with binary 
comparisons and Figure 16b shows an example of an acquired Live image of an established pattern 
which is to be verified. As is apparent, the pattern shown in Figure 16b contains numerous defects. 
20 Figures 17 through 26 show the defects which will and will not be detected by each of the respective 
ten comparisons listed in Appendix A. 

For example, Figure 17 demonstrates that the "If Template^ =1 then Live must = 1" 
comparison (labeled (1) in Appendix A) will identify the two horizontal breaks in the right hand 
vertical copper trace and will detect the pinhole adjacent the bottom of the same trace. However, as 
25 is also shown, the other defects, including >eck-down— 'extra copper" and "small feature missing- 
defects are not detected by this comparison. Figures 18 through 26 respectively show examples of 
the defects detected by the comparisons labeled (2) through (10) in Appendix A. 

It will be apparent to those of skill in the art that not all ten binary comparisons need be 
performed, nor do the comparisons need to be performed in a particular order. For example, as 
30 mentioned above, some of the binary comparisons are redundant when the background of the 

acquired image is merely the inverse of the foreground. Also, some pattern establishment processes 
are not subject to all types of defects. Accordingly, it is contemplated that the selection of 
appropriate binary comparisons and the order in which they are conducted will be readily ascertained 
by those employing the present invention. 
35 Figure 27 shows a block diagram of an implementation of the presently preferred binary 

embodiment of the instant invention. As shown in the Figure, the Live RAW image 350 from an 
appropriate camera 354 is transmitted to each of two binary correlation filters 358 and 362 and 
directly to the processor 366. Binary correlation filter 358 produces the Live,- R0DE data which is 
transmitted to Processor 366 and to binary correlation filter 370 which performs a dilate operation to • 
40 produce the Live 0PEN data which is also transmitted to Processor 366. Binary correlation filter 362 
produces the Live D1LATI f data which is transmitted to Processor 366 and to binary correlation filter 374 
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which performs an erode operation to produce the Live CL0SE data which is also transmitted to 
Processor 366. 

The TemplateRAw data 378, which can either be stored in memory or obtained from a 
previous verification operation, is transmitted to each of two binary correlation filters 382 and 386 
5 and directly to Processor 366. Binary correlation filter 382 produces the Templateo^ data and 
binary correlation filter 386 produces the Template DILATE data, each of which are also transmitted to 
Processor 366. Processor 366 performs the selected comparisons, combines the results of those 
comparisons and generates an appropriate defect detection output 390. 

As will be apparent, for clarity Figure 27 omits the buffers and other related functions which 
10 may be required to completely implement the system. However, Figure 28 shows an embodiment of 
the system of Figure 27 which has been constructed from commercially available Cyberscan® 
modules. In this Figure, eight Cyberscan® modules are employed, and specifically the system 
comprises: a CAM-DDC3-00 camera module 400; two BUF- 1000-00 buffer modules 404, 408; a 
BMF-8000-00 binary morphology module 412; a LUT-1000-00 look up table module 416; a BLB- 
15 1000-00 blob module 420; and a PCD-1000-00 display module 424. In this implementation, the 
Live RAW image, is acquired by a suitable camera 428 and any defects detected by the system are 
displayed to an operator on a conventional monitor 432. Output 436 may be used to provide detected 
defect results to a host computer via, for example, an RS-232 serial interface. 

The above-described embodiments are intended to be examples of the present invention and 
20 alterations and modifications may be effected thereto, by those of skill in the art, without departing 
from the scope of the invention which is defined solely by the claims appended hereto. 



APPENPIX A 

25 (1) If Template ER0DE = 1, then Live RAW must = 1. Used for detecting open defects in 
foreground, i.e. - pinholes or breaks in copper. Illustrated in Figure 17. 

(2) If Template DILATE = 0, then Live RAW must = 0. Used for detecting open defects in 
background, i.e. - extra copper in a void, a short circuit. Illustrated in Figure 18. 

(3) If Live^ = 1, then Template DILATE must = 1. Used for detecting open defects in 
30 background, i.e. - extra copper in a void, a short circuit. Illustrated in Figure 19. 

(4) If Live RAW = 0, then Template ER0DE must = 0. Used for detecting open defects in 
foreground, i.e. - pinholes in copper. Illustrated in Figure 20. 

(5) If Live£R 0DE = 1, then Template RAW must = 1. Used for detecting relatively large open 
defects in background, i.e. - large areas of extra copper, big short circuits. Illustrated in Figure 21. 

35 (6) If Live DILATE = 0, then Template RAW must = 0. Used for detecting relatively large open 
"defects in foreground and small missing foreground features, i.e. - large copper voids or missing 
small paths. Illustrated in Figure 22. 

(7) If Template RAW = 1, then Live DILATE must = 1. Used for detecting small missing foreground 
features, i.e. - copper pad missing. Illustrated in Figure 23. 
*0 (8) If Template RAW = 0, then Live EU0DE must = 0. Used for detecting small missing background 
features, i.e. - a drill hole in copper is missing. Illustrated in Figure 24. 
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(9) If Template CT0DE = 1 , then Live oreN must = 1 . Used for detecting narrowing of foreground 
features, i.e. - copper trace narrows, but is not broken. Often referred to as * neck-down'. 
Illustrated in Figure 25. 

(10) If Template DIUXE - 0, then Livec^ must - 0. Used for detecting narrowing of 

5 background features, i.e. - copper trace widens, but is not shorted. Often referred to as 'projection'. 
Illustrated in Figure 26. 
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We claim: 

1. A method of determining if an established pattern contains one or more defects, comprising 
the steps of: 

(i) defining a digital template image representing a desired pattern; 

(ii) acquiring a digital image of an established pattern; 

(iii) image processing at least one of said defined digital template and said acquired digital 
image to obtain at least first and second resultant images; 

(iv) performing at least one pair of comparisons comprising: a first comparison between the 
other of said defined template and said acquired digital image and said first resultant image; and a 
second comparison between the other of said defined template and said acquired digital image and 
said second resultant image; 

(v) evaluating the results of at least said at least one pair of comparisons to determine if said 
established pattern contains one or more defects. 

2. The method of claim 1 wherein said image processing in step (iii) comprises morphological 
image filter operations. 

3. The method of claim 2 wherein said processing in step (iii) said first resultant image is 
produced by an erode filter operation and said second resultant image is produced by a dilate filter 
operation. 

4. The method of claim 3 wherein said erode filter operation and said dilate filter operation are 
each implemented as rank value filters. 

5. The method of claim 1 wherein at least step (iv) is performed on a pixel by pixel basis. 

6. The method of claim 1 wherein each of said pair of comparisons in step (iv) are performed 
substantially in parallel. 

7. The method of claim 3 wherein: 

step (iii) comprises processing said digital template with each of an erode filter operation and 
a dilate filter operation to produce a resultant eroded template image and a resultant dilated template 
image and processing said acquired digital image with each of an erode filter operation and a dilate 
filter operation to produce a resultant eroded acquired image and a resultant dilated acquired image; 

step (iv) comprises performing a comparison between said acquired digital image and said 
resultant eroded template image, a comparison between said acquired digital image and said resultant 
dilated template image, a comparison between said defined template image and said resultant dilated 
acquired image and a comparison between said defined template image and said resultant eroded 
acquired image; and 

step (v) comprises evaluating the results of the four comparisons performed in step (iv) to 
determine if said established pattern contains one or more defects. 

The method of claim 7 wherein step (v) further comprises combining the results of the 
comparison between said acquired digital image and said resultant eroded template image and the 
results of the comparison between said acquired digital image said resultant dilated template image 
with a logical OR operation to determine if the established pattern contains at least one of an extra 
light feature or an extra dark feature. 

9. The method of claim 7 wherein step (v) further comprises combining the results of the 
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comparison between said defined template image and said resultant dilated acquired image and 
between said defined template image and said resultant eroded acquired image with a logical OR 
operation to determine if the established pattern is missing at least one of a light feature or a dark 
feature. 

10. The method of claim 8 wherein step (v) further comprises combining the results of the 
comparison between said defined template image and said resultant dilated acquired image and 
between said defined template image and said resultant eroded acquired image with a logical OR 
operation to determine if the established pattern is missing at least one of a light feature or a dark 
feature. 

11. The method of claim 1 wherein said at least first and second resultant images are stored in a 
memory means. 

12. The method of claim 7 wherein said resultant dilated template image and said resultant 
eroded template image are stored in a memory means. 

13. The method of claim 1 wherein said digital template and said acquired digital image are 
binary images. 

14. The method of claim 13 wherein said image processing in step (iii) comprises binary 
correlation image filter operations. 

15. The method of claim 14 wherein step (iii) comprises image processing at least one of said 
defined digital template and said acquired image with at least one of a dilate filter and an erode filter. 

16. The method of claim 15 wherein step (iii) further comprises image processing said acquired 
image with at least one of a series combination of a dilate filter and an erode filter and a series 
combination of an erode filter and a dilate filter. 

17. The method of claim 15 wherein step (iii) comprises each of said defined digital template and 
said acquired image being image processed with at least one of a dilate filter and an erode filter to 
obtain first and second resultant acquired images and first and second resultant template images and 
wherein step (iv) comprises performing at least one comparison between each of said first and second 
resultant acquired images and said defined template and performing at least one comparison between 
each of said first and second resultant template images and said acquired image. 

18. The method of claim 1 wherein said pair of comparisons of step (iv) are performed 
substantially simultaneously. 

19. The method of claim 7 wherein said comparisons of step (iv) are performed substantially 
simultaneously. 

20. The method of claim 17 wherein said comparisons of step (iv) are performed substantially 
simultaneously. 

21. The method of claim 1 wherein said template image is defined by acquiring a suitable image 
with a camera. 

22. A system to determine if an established pattern contains one or more defects, comprising: 
means to store a defined digital template for said pattern; 

means to acquire a digital image of said established pattern; 

image processing means to process at least one of said defined digital template and said 
acquired digital image to obtain at least first and second resultant images; 
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means to perform at least one pair of comparisons comprising a comparison between the 
other of said defined digital template and said acquired digital image and said first resultant image 
and a comparison between the other of said defined digital template and said acquired digital image 
and said second resultant image; and 

means to evaluate the results of said comparisons to determine if said established pattern 
contains one or more defects. 

23. The system of claim 22 wherein said image processing means comprises at least two 
morphological image filters. 

24. The system of claim 23 wherein one of said at least two morphological image filters is an 
erode filter and wherein another of said at least two morphological filters is a dilate filter. 

25. The system of claim 24 wherein each of said erode filter and said dilate filter are 
implemented as rank value filters. 

26. The system of claim 25 wherein said image processing means comprises at least two binary 
correlation filters. 

27. The system of claim 26 wherein said image processing means comprises at least two binary 
correlation filters. 

28. The system of claim 27 wherein at least one of said at least two binary correlation filters is a 
dilate filter and wherein another of said at least two binary correlation filters is an erode filter. 

29. The system of claim 28 wherein said image processing means further comprises at least one 
of a series combination of an erode and a dilate filter and a series combination of a dilate and an 
erode filter. 

30. The system of claim 22 wherein said image processing means processes said defined digital 
template and said first and second resultant images are stored in a memory. 

31. The system of claim 22 wherein said defined digital template is a previously acquired digital 
image. 

32. The system of claim 22 wherein said image processing means processes said at least one of 
said defined digital template and said acquired digital image in parallel to obtain substantially 
simultaneously said at least first and second resultant images. 
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